Information retrieval (IR) systems such as web search engines are becoming increasingly prominent as the Internet grows in both amount of information indexed and the number of users. While gains are being made to improve the performance and effectiveness of these systems, the majority of IR systems are static. These IR systems are constructed by chaining together a fixed set of IR techniques such as query formation, query evaluation, precision improvement, recall improvement, clustering, and results visualization. Once an IR system is completed and put into operation, the same sequence of operations is performed on every query processed. The static nature of IR systems reflects the methods used to evaluate new IR techniques. An IR technique is typically tested offline to determine if the technique is reliable enough to be included in an IR system. In these experiments, a test collection including a set of queries and a set of documents is used. Each document is tagged with relevance judgments for each query in the test collection. A document is tagged as relevant if the document satisfies the information needed posed by the query.